Precision Education: The Future of Lifelong Learning in Medicine.

ABSTRACT: The goal of medical education is to produce a physician workforce capable of delivering high-quality equitable care to diverse patient populations and communities. To achieve this aim amidst explosive growth in medical knowledge and increasingly complex medical care, a system of personalized and continuous learning, assessment, and feedback for trainees and practicing physicians is urgently needed. In this perspective, the authors build on prior work to advance a conceptual framework for such a system: precision education (PE).PE is a system that uses data and technology to transform lifelong learning by improving personalization, efficiency, and agency at the individual, program, and organization levels. PE "cycles" start with data inputs proactively gathered from new and existing sources, including assessments, educational activities, electronic medical records, patient care outcomes, and clinical practice patterns. Through technology-enabled analytics , insights are generated to drive precision interventions . At the individual level, such interventions include personalized just-in-time educational programming. Coaching is essential to provide feedback and increase learner participation and personalization. Outcomes are measured using assessment and evaluation of interventions at the individual, program, and organizational levels, with ongoing adjustment for repeated cycles of improvement. PE is rooted in patient, health system, and population data; promotes value-based care and health equity; and generates an adaptive learning culture.The authors suggest fundamental principles for PE, including promoting equity in structures and processes, learner agency, and integration with workflow (harmonization). Finally, the authors explore the immediate need to develop consensus-driven standards: rules of engagement between people, products, and entities that interact in these systems to ensure interoperability, data sharing, replicability, and scale of PE innovations.

A Theoretical Foundation to Inform the Implementation of Precision Education and Assessment.

ABSTRACT: Precision education (PE) uses personalized educational interventions to empower trainees and improve learning outcomes. While PE has the potential to represent a paradigm shift in medical education, a theoretical foundation to guide the effective implementation of PE strategies has not yet been described. Here, the authors introduce a theoretical foundation for the implementation of PE, integrating key learning theories with the digital tools that allow them to be operationalized. Specifically, the authors describe how the master adaptive learner (MAL) model, transformative learning theory, and self-determination theory can be harnessed in conjunction with nudge strategies and audit and feedback dashboards to drive learning and meaningful behavior change. The authors also provide practical examples of these theories and tools in action by describing precision interventions already in use at one academic medical center, concretizing PE's potential in the current clinical environment. These examples illustrate how a firm theoretical grounding allows educators to most effectively tailor PE interventions to fit individual learners' needs and goals, facilitating efficient learning and ultimately improving patient and health system outcomes.

Precision Medical Education.

Medical schools and residency programs are increasingly incorporating personalization of content, pathways, and assessments to align with a competency-based model. Yet, such efforts face challenges involving large amounts of data, sometimes struggling to deliver insights in a timely fashion for trainees, coaches, and programs. In this article, the authors argue that the emerging paradigm of precision medical education (PME) may ameliorate some of these challenges. However, PME lacks a widely accepted definition and a shared model of guiding principles and capacities, limiting widespread adoption. The authors propose defining PME as a systematic approach that integrates longitudinal data and analytics to drive precise educational interventions that address each individual learner's needs and goals in a continuous, timely, and cyclical fashion, ultimately improving meaningful educational, clinical, or system outcomes. Borrowing from precision medicine, they offer an adapted shared framework. In the P4 medical education framework, PME should (1) take a proactive approach to acquiring and using trainee data; (2) generate timely personalized insights through precision analytics (including artificial intelligence and decision-support tools); (3) design precision educational interventions (learning, assessment, coaching, pathways) in a participatory fashion, with trainees at the center as co-producers; and (4) ensure interventions are predictive of meaningful educational, professional, or clinical outcomes. Implementing PME will require new foundational capacities: flexible educational pathways and programs responsive to PME-guided dynamic and competency-based progression; comprehensive longitudinal data on trainees linked to educational and clinical outcomes; shared development of requisite technologies and analytics to effect educational decision-making; and a culture that embraces a precision approach, with research to gather validity evidence for this approach and development efforts targeting new skills needed by learners, coaches, and educational leaders. Anticipating pitfalls in the use of this approach will be important, as will ensuring it deepens, rather than replaces, the interaction of trainees and their coaches.

Artificial Intelligence Screening of Medical School Applications: Development and Validation of a Machine-Learning Algorithm.

PURPOSE: To explore whether a machine-learning algorithm could accurately perform the initial screening of medical school applications. METHOD: Using application data and faculty screening outcomes from the 2013 to 2017 application cycles (n = 14,555 applications), the authors created a virtual faculty screener algorithm. A retrospective validation using 2,910 applications from the 2013 to 2017 cycles and a prospective validation using 2,715 applications during the 2018 application cycle were performed. To test the validated algorithm, a randomized trial was performed in the 2019 cycle, with 1,827 eligible applications being reviewed by faculty and 1,873 by algorithm. RESULTS: The retrospective validation yielded area under the receiver operating characteristic (AUROC) values of 0.83, 0.64, and 0.83 and area under the precision-recall curve (AUPRC) values of 0.61, 0.54, and 0.65 for the invite for interview, hold for review, and reject groups, respectively. The prospective validation yielded AUROC values of 0.83, 0.62, and 0.82 and AUPRC values of 0.66, 0.47, and 0.65 for the invite for interview, hold for review, and reject groups, respectively. The randomized trial found no significant differences in overall interview recommendation rates according to faculty or algorithm and among female or underrepresented in medicine applicants. In underrepresented in medicine applicants, there were no significant differences in the rates at which the admissions committee offered an interview (70 of 71 in the faculty reviewer arm and 61 of 65 in the algorithm arm; P = .14). No difference in the rate of the committee agreeing with the recommended interview was found among female applicants (224 of 229 in the faculty reviewer arm and 220 of 227 in the algorithm arm; P = .55). CONCLUSIONS: The virtual faculty screener algorithm successfully replicated faculty screening of medical school applications and may aid in the consistent and reliable review of medical school applicants.

Exploiting the power of information in medical education.

The explosion of medical information demands a thorough reconsideration of medical education, including what we teach and assess, how we educate, and whom we educate. Physicians of the future will need to be self-aware, self-directed, resource-effective team players who can synthesize and apply summarized information and communicate clearly. Training in metacognition, data science, informatics, and artificial intelligence is needed. Education programs must shift focus from content delivery to providing students explicit scaffolding for future learning, such as the Master Adaptive Learner model. Additionally, educators should leverage informatics to improve the process of education and foster individualized, precision education. Finally, attributes of the successful physician of the future should inform adjustments in recruitment and admissions processes. This paper explores how member schools of the American Medical Association Accelerating Change in Medical Education Consortium adjusted all aspects of educational programming in acknowledgment of the rapid expansion of information.

Signatures of medical student applicants and academic success.

The acceptance of students to a medical school places a considerable emphasis on performance in standardized tests and undergraduate grade point average (uGPA). Traditionally, applicants may be judged as a homogeneous population according to simple quantitative thresholds that implicitly assume a linear relationship between scores and academic success. This 'one-size-fits-all' approach ignores the notion that individuals may show distinct patterns of achievement and follow diverse paths to success. In this study, we examined a dataset composed of 53 variables extracted from the admissions application records of 1,088 students matriculating to NYU School of Medicine between the years 2006-2014. We defined training and test groups and applied K-means clustering to search for distinct groups of applicants. Building an optimized logistic regression model, we then tested the predictive value of this clustering for estimating the success of applicants in medical school, aggregating eight performance measures during the subsequent medical school training as a success factor. We found evidence for four distinct clusters of students-we termed 'signatures'-which differ most substantially according to the absolute level of the applicant's uGPA and its trajectory over the course of undergraduate education. The 'risers' signature showed a relatively higher uGPA and also steeper trajectory; the other signatures showed each remaining combination of these two main factors: 'improvers' relatively lower uGPA, steeper trajectory; 'solids' higher uGPA, flatter trajectory; 'statics' both lower uGPA and flatter trajectory. Examining the success index across signatures, we found that the risers and the statics have significantly higher and lower likelihood of quantifiable success in medical school, respectively. We also found that each signature has a unique set of features that correlate with its success in medical school. The big data approach presented here can more sensitively uncover success potential since it takes into account the inherent heterogeneity within the student population.

Extended Reality in Medical Education: Driving Adoption through Provider-Centered Design.

Simulation is a widely used technique for medical education. Due to decreased training opportunities with real patients, and increased emphasis on both patient outcomes and remote access, demand has increased for more advanced, realistic simulation methods. Here, we discuss the increasing need for, and benefits of, extended (virtual, augmented, or mixed) reality throughout the continuum of medical education, from anatomy for medical students to procedures for residents. We discuss how to drive the adoption of mixed reality tools into medical school's anatomy, and procedural, curricula.

The Time Is Now: Using Graduates' Practice Data to Drive Medical Education Reform.

Medical educators are not yet taking full advantage of the publicly available clinical practice data published by federal, state, and local governments, which can be attributed to individual physicians and evaluated in the context of where they attended medical school and residency training. Understanding how graduates fare in actual practice, both in terms of the quality of the care they provide and the clinical challenges they face, can aid educators in taking an evidence-based approach to medical education. Although in their infancy, efforts to link clinical outcomes data to educational process data hold the potential to accelerate medical education research and innovation. This approach will enable unprecedented insight into the long-term impact of each stage of medical education on graduates' future practice. More work is needed to determine best practices, but the barrier to using these public data is low, and the potential for early results is immediate. Using practice data to evaluate medical education programs can transform how the future physician workforce is trained and better align continuously learning medical education and health care systems.

The Role for Virtual Patients in the Future of Medical Education.

The medical education community is working-across disciplines and across the continuum-to address the current challenges facing the medical education system and to implement strategies to improve educational outcomes. Educational technology offers the promise of addressing these important challenges in ways not previously possible. The authors propose a role for virtual patients (VPs), which they define as multimedia, screen-based interactive patient scenarios. They believe VPs offer capabilities and benefits particularly well suited to addressing the challenges facing medical education. Well-designed, interactive VP-based learning activities can promote the deep learning that is needed to handle the rapid growth in medical knowledge. Clinically oriented learning from VPs can capture intrinsic motivation and promote mastery learning. VPs can also enhance trainees' application of foundational knowledge to promote the development of clinical reasoning, the foundation of medical practice. Although not the entire solution, VPs can support competency-based education. The data created by the use of VPs can serve as the basis for multi-institutional research that will enable the medical education community both to better understand the effectiveness of educational interventions and to measure progress toward an improved system of medical education.

NYU3T: teaching, technology, teamwork: a model for interprofessional education scalability and sustainability.

Interprofessional education is a critical precursor to effective teamwork and the collaboration of health care professionals in clinical settings. Numerous barriers have been identified that preclude scalable and sustainable interprofessional education (IPE) efforts. This article describes NYU3T: Teaching, Technology, Teamwork, a model that uses novel technologies such as Web-based learning, virtual patients, and high-fidelity simulation to overcome some of the common barriers and drive implementation of evidence-based teamwork curricula. It outlines the program's curricular components, implementation strategy, evaluation methods, and lessons learned from the first year of delivery and describes implications for future large-scale IPE initiatives.

New directions in e-learning research in health professions education: Report of two symposia.

BACKGROUND: The use of Computer Assisted Instruction (CAI) is rising across health professions education. Research to date is of limited use in guiding the implementation and selection of CAI innovations. AIMS: In the context of two symposia, systemic reviews were discussed that evaluate literature in Internet-based learning, Virtual Patients, and animations. Each session included a debate with the goal of reaching consensus on best current practices and future research. METHODS: Thematic analysis of the discussions was performed to arrange the questions by theme, eliminate redundancy, and craft them into a cohesive narrative. RESULTS: The question analysis revealed that there are clear advantages to the use of CAI, and that established educational theories should certainly inform the future development and selection of CAI tools. Schools adopting CAI need to carefully consider the benefits, cost, available resources, and capacity for teachers and learners to accept change in their practice of education. Potential areas for future research should focus on the effectiveness of CAI instructional features, integration of e-learning into existing curricula and with other modalities like simulation, and the use of CAI in assessment of higher-level outcomes. CONCLUSIONS: There are numerous opportunities for future research and it will be important to achieve consensus on important themes.

Enhanced virtual microscopy for collaborative education.

BACKGROUND: Curricular reform efforts and a desire to use novel educational strategies that foster student collaboration are challenging the traditional microscope-based teaching of histology. Computer-based histology teaching tools and Virtual Microscopes (VM), computer-based digital slide viewers, have been shown to be effective and efficient educational strategies. We developed an open-source VM system based on the Google Maps engine to transform our histology education and introduce new teaching methods. This VM allows students and faculty to collaboratively create content, annotate slides with markers, and it is enhanced with social networking features to give the community of learners more control over the system. RESULTS: We currently have 1,037 slides in our VM system comprised of 39,386,941 individual JPEG files that take up 349 gigabytes of server storage space. Of those slides 682 are for general teaching and available to our students and the public; the remaining 355 slides are used for practical exams and have restricted access. The system has seen extensive use with 289,352 unique slide views to date. Students viewed an average of 56.3 slides per month during the histology course and accessed the system at all hours of the day. Of the 621 annotations added to 126 slides 26.2% were added by faculty and 73.8% by students. The use of the VM system reduced the amount of time faculty spent administering the course by 210 hours, but did not reduce the number of laboratory sessions or the number of required faculty. Laboratory sessions were reduced from three hours to two hours each due to the efficiencies in the workflow of the VM system. CONCLUSIONS: Our virtual microscope system has been an effective solution to the challenges facing traditional histopathology laboratories and the novel needs of our revised curriculum. The web-based system allowed us to empower learners to have greater control over their content, as well as the ability to work together in collaborative groups. The VM system saved faculty time and there was no significant difference in student performance on an identical practical exam before and after its adoption. We have made the source code of our VM freely available and encourage use of the publically available slides on our website.

Computerized virtual patients in health professions education: a systematic review and meta-analysis.

PURPOSE: Educators increasingly use virtual patients (computerized clinical case simulations) in health professions training. The authors summarize the effect of virtual patients compared with no intervention and alternate instructional methods, and elucidate features of effective virtual patient design. METHOD: The authors searched MEDLINE, EMBASE, CINAHL, ERIC, PsychINFO, and Scopus through February 2009 for studies describing virtual patients for practicing and student physicians, nurses, and other health professionals. Reviewers, working in duplicate, abstracted information on instructional design and outcomes. Effect sizes were pooled using a random-effects model. RESULTS: Four qualitative, 18 no-intervention controlled, 21 noncomputer instruction-comparative, and 11 computer-assisted instruction-comparative studies were found. Heterogeneity was large (I²>50%) in most analyses. Compared with no intervention, the pooled effect size (95% confidence interval; number of studies) was 0.94 (0.69 to 1.19; N=11) for knowledge outcomes, 0.80 (0.52 to 1.08; N=5) for clinical reasoning, and 0.90 (0.61 to 1.19; N=9) for other skills. Compared with noncomputer instruction, pooled effect size (positive numbers favoring virtual patients) was -0.17 (-0.57 to 0.24; N=8) for satisfaction, 0.06 (-0.14 to 0.25; N=5) for knowledge, -0.004 (-0.30 to 0.29; N=10) for reasoning, and 0.10 (-0.21 to 0.42; N=11) for other skills. Comparisons of different virtual patient designs suggest that repetition until demonstration of mastery, advance organizers, enhanced feedback, and explicitly contrasting cases can improve learning outcomes. CONCLUSIONS: Virtual patients are associated with large positive effects compared with no intervention. Effects in comparison with noncomputer instruction are on average small. Further research clarifying how to effectively implement virtual patients is needed.

Longitudinal research databases in medical education: facilitating the study of educational outcomes over time and across institutions.

Many education research questions cannot be answered using participants from one institution or short periods of follow-up. In response to societal demands for accountability and evidence of effectiveness, new models of research must be developed to study the outcomes of educational activities. Following the 2007 Millennium Conference on Medical Education Research, organizers assigned a task force to explore the use of longitudinal databases in education research. This article summarizes the task force's findings. Similar to the Framingham studies in clinical medicine, longitudinal databases assemble prospectively collected information to retrospectively answer questions of interest. Many studies using such databases have been published. The task force identified three general approaches to database-type research. First, institutions can obtain identified information from existing sources, link it with school-specific information and other identified information, deidentify it, and merge it with similar information from other collaborating schools. Second, researchers can obtain from existing sources deidentified information on large samples and explore associations within this dataset. Third, investigators can design and implement databases to prospectively collect trainee information over time and across multiple institutions for the purpose of education research. Although costly, such comprehensive, purpose-built databases would ensure the availability of information needed to answer a variety of medical education research questions. Millennium Conference participants believed that stakeholders should explore the funding and development of such prospective databases. In the meantime, education researchers should use existing sources of individualized learner data to better understand how to develop competent, compassionate clinicians.

Health information technology and the medical school curriculum.

Medical schools must teach core biomedical informatics competencies that address health information technology (HIT), including explaining electronic medical record systems and computerized provider order entry systems and their role in patient safety; describing the research uses and limitations of a clinical data warehouse; understanding the concepts and importance of information system interoperability; explaining the difference between biomedical informatics and HIT; and explaining the ways clinical information systems can fail. Barriers to including these topics in the curricula include lack of teachers; the perception that informatics competencies are not applicable during preclinical courses and there is no place in the clerkships to teach them; and the legal and policy issues that conflict with students' need to develop skills. However, curricular reform efforts are creating opportunities to teach these topics with new emphasis on patient safety, team-based medical practice, and evidence-based care. Overarching HIT competencies empower our students to be lifelong technology learners.